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Abstract 

At Kennedy Space Center (KSC), an important use of Discrete Event Simulation (DES) addresses ground 
operations of missions to space. DES allows managers, scientists and engineers to assess the number of 
missions KSC can complete on a given schedule within different facilities, the effects of various 
configurations of resources and detect possible problems or unwanted situations. For fifteen years, DES has 
supported KSC efficiency, cost savings and improved safety and performance. The dense and abstract DES 
data, however, proves difficult to comprehend and, NASA managers realized, is subject to 
misinterpretation, misunderstanding and even, misuse. In summer 2008, KSC developed and implemented 
a NASA Exploration Systems Mission Directorate (ESMD) project based on the premise that visualization 
could enhance NASA’s understanding and use of DES. 


The Premise 

Expanding the use and utility of simulation in 
creating and maintaining massively complex 
systems demands advances in technology, as 
well as interoperability among data, simulation, 
hardware and humans. Less obviously, perhaps, 
it requires communicating findings in a clear, 
consistent and effective manner. We must ensure 
that others can see, accurately and immediately, 
exactly what we want the simulation to show 
them. Our goal is to elicit insight that supports 
understanding and better decision-making. 
Through inattention or ignorance of the 
challenges inherent in visual communication, we 
may, instead, confuse or discourage the viewer. 
This is important when simulation addresses 
complex systems and serious hazards and 
situations involving time, money, and potential 
loss of equipment, missions and lives. This also 
suggests that universally accepted standards for 
visual communication of such simulation are not 
merely a nicety but a necessity. [1] At the John 
F. Kennedy Space Center (KSC) in Florida, 
simulation supports ground processing that may 
include check-out, assembly, test, integration, 
evaluation, training, launch and landing. 

Problem as Given 

The project focused on the KSC ground 
processing DES for the NASA Constellation 


program and its two spaceships in development: 
ARES I and ARES V. Ares I is the vehicle that 
will allow Orion, the crew exploration vehicle, to 
return humans to the Moon to stay and, then, in 
time, go on to Mars and beyond. Ares V, 
Constellation’s cargo launch component, is a 
heavy lift spacecraft to support human and 
automated missions. 

Simulation, including discrete event simulation 
(DES), is essential at every stage of the 
operation. We believed that visualization would 
add to our understanding of DES that 
mathematically represents KSC ground 
processing as a chronological sequence of 
events. With DES, KSC engineers and scientists 
can simulate ground operations of missions to 
Space, enable them to assess facility utilization, 
analyze systems and plan operations as well as 
experiment and predict potential outcomes. This 
is designed to support planning, operations, 
testing and improvement. However, the DES 
model of ground processing operations over 15 
years is dense and abstract. It is difficult to 
understand, even by its creators. The potential 
for misunderstanding is a serious concern. Data 
visualization, it was believed, could play an 
important role in making data more useful and 
accessible to a broader audience. 


Recognizing that underutilization of DES, 
despite its reputation for high value, rested on 
problems related to failure to communicate, the 
organization made a simple request to the KSC 
DES Simulation Visualization team (DES/VIZ), 
“make Kennedy Space Center (KSC) Discrete 
Event Simulation (DES) more visual.” 

The DES/VIZ team, sponsored by the NASA 
ESMD as a summer intern project, accepted the 
premise that visualization would add to critically 
important understanding of DES. 

Because KSC is so large- 10 times the size of 
Manhattan or 219 square miles — any attempt to 
show the entire flow of ground processing 
activities presents an image of an army of tiny 
frantic ants running all over the place. This was 
never a serious choice for the project. NASA 
can, at far less cost or effort, show the flow of 
activities on a map or in a film or animation. 
Depiction of entities and events or their location 
fails to indicate the depth of information that 
DES could and should make more available and 
useful for decision-making. 

The final scenario does open with a map of 
KSC, but it highlights only the key facilities that 
the simulation addresses including the Vehicle 
Assembly Building (VAB), the Rotation, 
Processing and Surge Facility (RPSF) and the 
Launch Pad. 

When adding visualization to DES, others 
traditionally insert simple images (building, 
people, animal, equipment or vehicle) to a 
particular event stage. The team departed from 
such an approach and, instead, worked on the 
problem of depicting the data hidden within 15 
years worth of simulation. This is easier said 
than done. The data is dense, and we understood 
that abstract concepts are more easily 
misinterpreted than concrete ones. The premise 
of the DES/VIZ project was to do the job more 
concretely and, therefore, better with the ultimate 
goal of more frequent and effective use. 

The Problem as Understood 

After redefining the problem to “depict data 
within DES to improve decision-making 
capability,” the DES/VIZ team worked 
specifically on creating a 3-D visualization of 
data embedded in NASA’s DES Exploration 
Launch and Landing Analysis (ELLA). Ella is a 
DES model created in Rockwell Arena. Some 
believed the size and complexity of ELLA’s data 


made this an impossible task. They suggested 
reducing expectations by reframing the project 
merely as a study of the problem. The DES/VIZ 
team rejected this course and, instead, 
determined to , follow an innovative results- 
driven approach in a process of framing the 
issue, optimizing their limited resources and 
substantiating their approach. [2] In its initial 
research, the team found no example, no findings 
nor useful methods to simply copy. With only 
50 days to do the work, the DES/VIZ team faced 
a steep learning curve. In almost no time, the 
team had to learn about KSC ground processing, 
DES, game technology and NASA’s Distributed 
Observer Network (DON) and understand 
ELLA. 

The ELLA output files were so huge (hundreds 
of thousands of lines) that just opening and 
trying to browse in them was a challenge. 
Lacking documentation, the technical lead built a 
custom tool to facilitate pattern identification. 
This opened a means for reading the events and 
led to a plan to parse the output and create a 
sequence to playback in the 3D visualization 
tool. 

Facing uncertainty and ambiguity, the team 
asked questions. They added consultants and 
colleagues across disciplines and organizations. 
These included at least 12 specialists and 
generalists. Some played multiple roles: 
customer, technical adviser, discrete event 
simulation developer and analyst, modeler, 
engineer, computer scientist, software 
development supervisor, human factors and 
simulation engineer, integration expert, 
strategist, film maker, scenario writer, project 
manager, game development technologist and 
game player. 

People with different viewpoints brought 
important and critically different talents to the 
task. Various studies link success in reducing 
uncertainty with multiple sources of knowledge, 
skills and experience. Eric Raymond [3] 
suggested that a large base of distributed users 
could help improve design outcomes rapidly and 
become indispensable co-developers. Each views 
problems and opportunities “with a slightly 
different perceptual set and analytic toolkit.” The 
literature, Herstatt and others [4] also speak not 
only of value, but of the leadership opportunity 
awaiting organizations that actively create and 
use such multidisciplinary teams. Few have 
done so. 


The necessary expertise to solve difficult 
problems cannot easily be found in any one 
institutional setting. The DES/VIZ experience 
confirmed the importance of early multi- 
disciplinary interest and input that studies say 
can produce improved performance, save time, 
reduce later unplanned changes that add to cost 
and obstruct opportunities for success. Goguen 
states: "Ambiguity and informality facilitate 
gradual evolution of requirements for complex 
systems without forcing too early resolution of 
conflicts and ambiguities inherent in the system, 
avoids prejudging tradeoffs (cost, speed, 
functionality). [5] An analogy might be the 
journalists who spend three- fourths of their time 
writing the first or lead sentence. Once that 
difficulty is mastered, the copy follows logically, 
economically and with a clear focus. 

The team sought help and worked cooperatively 
with all who came in and participated in the 
friendly chaos of Room 2111 in the KSC 
Operations and Checkout Building (O&C) where 
they soon covered white boards, walls and, even, 
floors, with research results, ideas, quotations, 
and questions. New problems require trial and 
error. The point is to do so quickly, learn and 
move on. This became the pattern of work. 

They experimented, changed direction several 
times and created models, charts and rapid 
prototypes showing potential answers. As a 
framework for sharing perception, these 
probably helped dampen biases especially from 
those who thought the project could not be done. 

Research on best practices in integrating 
visualization into DES is absent. The team 
invented and chronicled how to do this work. 
They defined and faced many challenges and 
discovered both problems and opportunities. 
They kept notes and built charts showing the 
“Steps to make DES Visible in DON” and “The 
Path of Flight Hardware in the Exploration 
Launch and Landing Analysis (ELLA)” as well 
as numerous models and simulations. Iteration 
and loop backs appear to have been important 
because the team considered, tested and 
eliminated many options. Using simulation 
during upfront development guided the design 
toward optimally satisfying performance. 

The customer recognized the need for a new 
product and supported a changing custom 
solution as the team learned more. The literature 
reports that such support is meaningful and, 


even, critical. [6] However management 
involvement may not be beneficial and may 
damage creativity by constantly changing goals. 
[7] Interesting projects inspire added ideas. 
Indeed, such enthusiasm nearly derailed progress 
on this project by suggesting a totally new-and 
probably useful and exciting but irrelevant- 
project. Wisdom prevailed after strong 
opposition. 

Framing the task 

The KSC DES analysis team, concerned about 
potential under-planning that could result in 
over-utilization of resources, posed as a problem 
that the Crew Exploration Vehicle (CEV) 
transporter was broken and out-of service for 200 
days. The transporter, a one-of-a-kind resource 
in ESMD ground operations, has the job of 
moving the spacecraft from facility to facility for 
processing and ultimately to the pad for launch 
into Space. There is no back up. What impact 
would such a breakdown have on the 123-day 
schedule for ground processing? 

To the surprise of the analysts, the DES 
indicated a domino effect almost obscured by the 
mass of data. DES data told them, but obscurely, 
that delays shall occur, and they will be very bad. 

The DES/VIZ team, working with the DES 
analysts, constantly reminded themselves that 
they only had to “crack” one small amount of 
simulation data to establish a pathway for future 
advancements in depicting pertinent DES 
information. By focusing on a small example of 
valuable DES data, the team never lost sight of 
the mission and seemed to breeze past formal 
requirements. 

A Harvard study of Innovation [8] provided a 
partial framework. It was clear that whatever the 
team did had to be simple, frugal and straight- 
forward, providing quick results that would be 
easy for a wide constituency to understand and 
voluntarily accept. The project also included 
deliberate attention to creativity standards. These 
are similar to those for innovation but have 
differences that could produce tensions and 
require integration. These are: the importance of 
openness to options, delaying closure, attention 
to discovery, questioning and engagement as 
well as to the pragmatic need for change and 
redefinition of problems. [6] Integration of these 
ideas combining innovation and creativity, 
proved relatively easy, both doable and valuable. 


Optimizing options 

The DES/VIZ team found no easy answers. The 
data and technology remained a challenge. 
Nonetheless constant questioning did produce 
possibilities. Planning and openness to ideas 
suggested more ideas that eventually worked and 
helped “crack” the essential problem. 

The decision to depict DES data grew into the 
refined determination to show the relationship 
among delays involving the Crew Assembly 
(CEV) transporter, Solid Rocket Booster (SRB) 
segment inventory in the Rotation, Processing 
and Surge Facility (RPSF) and aft skirt 
processing and stacking in the Vehicle Assembly 
Building (VAB). The aft skirt forms the base of 
the solid rocket booster. After refurbishment and 
assembly, NASA moves the aft skirt hardware to 
the RPSF where SRB segments, loaded with 
solid propellant, arrive by rail for ground 
processing. NASA next moves that assembly to 
the nearby VAB for remaining integration and 
preparation for the move to the launch pad. 

The model assumes an extreme case, a 
breakdown requiring 200 days to fix. Late in the 
project (day 42 of 50), the data finally revealed a 
delay of 666 days in delivery of the aft skirt to 
the VAB. This suggested a clear problem for 
investigation. After this delivery, the stacking in 
the VAB would take 257 days. The vehicle 
would leave for the Pad on Day 1123 and be on 
the Pad for 13 days prior to launch. The latter is 
normal. Nothing else is. Processing is supposed 
to take 123 days. Because of the 200-day delay 
with the transporter, the launch would be 
delayed, unacceptably, over 1000 days. 

These startling numbers demand attention, and it 
is visualization that makes them clear. Even DES 
analysts assessing the abstract DES data had 
difficulty evaluating the relationship between the 
transporter and the length of the delays. Without 
the visualization, some might well assume that a 
delay of 200 days in the use of the single 
transporter would mean a delay of little more 
than 200 days. 

The DES scenario traces what might seem 
counterintuitive to many managers. If the CEV 
transporter is in use 100% of the time, some may 
see this as efficient and cost-effective. It is not. 
Such use is over-use. It allows no time for 
maintenance or contingencies (unavailability of 
human resources, breakdown either in the 
schedule or, as in this case, equipment). Ground 


processing personnel prefer 75% usage to allow 
for opportunities and contingencies. 

The project depicts two essential pieces of data: 
utilization and cumulative time. The design uses 
a gauge with a pointer to indicate when the 
utilization is nominal and when it is not (under 
or over-utilized). For utilization the team showed 
nominal use as green. Everyone could see this as 
a positive piece of data. Some confusion arose 
when the design showed both under-utilization 
and over-utilization as “red” to indicate problems 
(with yellow as warnings on either side). Over- 
utilization is not, of course, exactly the same as 
under-utilization. Under-utilization may be 
wasteful, but it does not cause delays that impact 
the mission. 

The current design shows under-utilization as 
blue. This choice reflects the conclusion that 
under-utilization is a ’’cooler”, less problematic 
issue, and blue is a “cool” color in the spectrum. 
Nominal, in the simulation, remains as the color 
green. Over-utilization is now clearly and solely 
indicated in the warmest color in the spectrum, 
red, which universally indicates danger or 
“stop.” This differentiation seems obvious in 
retrospect, but it is an example of how, as use of 
visualization expands in simulation, we must 
develop both awareness (leading to standards) 
and behavior (based on standards) that result in 
correct action taken correctly. 

The team considered several means to visualize 
the cumulative days at each major stage of the 
simulation. They selected a clock image with 
relevant numbers against a background that turns 
red when there is a delay. This is consistent with 
the use of red solely for over-utilization. This 
may, in future, be a percentage chart. 

What the data makes clear is that if the 
transporter is over-used (or out-of-service for 
any reason), delays shall occur. The data does 
not tell why the ground processing took more 
than 1000 days. It suggests the need to look 
again more closely to see what else is happening 
regarding the RPSF, solid rocket boosters, the aft 
skirt and the VAB. It reminds us to challenge the 
model. The latter reflects the admonition of 
simulation scientist, John Sternman who has 
declared, “models lie... because we cannot 
validate any model for truthfulness.” [9]) 

From the beginning, the plan had been to display 
the work on NASA’s Distributed Observer 


Network (DON). DON had been built for ESMD 
using the Torque Game Engine from Garage 
Games. Unfortunately, more than midway 
through the project, the team understood that in 
order to achieve DON’S amazing ability to 
brilliantly display advanced 3-D simulation, the 
DON team had disabled a number of Torque 
game technology elements. These, desirable to 
the DES/VIZ project, included text, health meter, 
flashing lights, moving lights, a window for an 
orientation map, object shaking, object moving 
and camera freedom. Without them, the team had 
fewer visualization options. 

For Plan B, the team laid out a matrix and 
selected Torque, the original game engine that 
powers DON. The experience with DON, 
however, was useful since it did open the link 
between Ella and 3-D visualization capability. 
This led to the technical lead creating a 3_D 
visualization tool using MilkShape 3d which can 
create models with both a very light polygon 
count and an acceptable appearance. 

The DES/VIZ team simplified the component 
models as compared to the advanced 3-D models 
NASA displays on DON for two reasons. One, 
the DES data is so large that it was important 
that the model not compete for memory. Two, 
model detail and fidelity were not issues. The 
goal was to enable the viewer to see and 
understand the data that the DES contains. 
Generic and simple models helped to both clarify 
complex information and not overtax the system. 

Torque has the capabilities the project needs to 
show the data. Moreover, its Torque script, 
similar to C language, allowed an easy learning 
curve, leveraging the technical lead’s C 
programming skills 

Torque is not, however, designed to show time, 
as DON can, so it was not a perfect fit. A new 
version of DON became available but, with 15 
days remaining, the team did not have time to 
learn it. Torque was the best available answer. 
Fortunately, anything the DES/VIZ team did in 
Torque would be transferable to later versions of 
DON based on Torque 


Substantiating the Idea 

The team tracked and stored its notes regarding 
challenges, concepts, resources, technical notes, 
expert consultants, descriptions, online links, 
images, models, simulation and other 


information in a strategic archive. This will serve 
as an informal design “notebook” for future DES 
visualization of critical information including 
adaptation to a future DON or new tools. This 
strategic DES Visualization archive includes the 
charts, simulation screen shots, sources of 
information and other materials referenced in 
this paper. 

The project provides a pathway for future efforts 
to identify and show other examples of specific 
data in order to not only maintain but improve 
operations and make DES more understandable, 
more useful and used effectively. 

The DES data alone tells us, but obscurely, how 
bad the. delay could be. Visualization enables 
people to see it, concretely, for themselves. This 
demonstration is just the beginning of future 
opportunities, including a use case in 
development, to correct and improve operations 
using clear, tested and consistent depictions of 
the DES information. 

The team determined the future value of a 
nominal model showing the CEV processing as 
123 days with the Utilization Gauge showing 
normal (green) and the clock also green. The 
team sees value in gathering more data to depict 
visually the problems that led to the 666-day 
delay in the delivery of the aft skirt. This is a 
mystery worth solving. It would be advantageous 
if DES/Visualization could support a process 
reducing the 123-day ground-processing 
schedule to increase cost-effectiveness and 
improve performance. 

The DES visualization project provided the 
DES/VIZ team with an opportunity to work on a 
practical problem of real value to the space 
exploration program and learn by doing. They 
accepted technology, time and learning curve 
constraints as they defined, gathered, tested and 
systematically maintained a variety of 
information resources. Throughout the project, 
the team members initiated and maintained a 
method of light but focused planning, identified 
and regularly revisited outcome, criteria, and 
priorities as they explored uncertainties and 
consulted within a wide range ’ of 
multidisciplinary individuals. 

Concentrating considerable time and energy to 
an upfront planning effort paid off. Many 
companies now spend up to 60% of their 
engineering budget before detailed design 
begins. Such an approach informed the approach 


on this project This emphasis on early design 
provided many opportunities for innovation and 
supported the move from the problem as given- 
“add visualization to DES”-- to the more 
substantive- “depict DES information visually 
in order to improve decision-making capability.” 

The results demonstrated that careful 

visualization of the data commands attention as 
it enables others to see concretely that a 200-day 
delay would grow overtime to a launch delay of 
more than 1000 days. This successful DES/VIZ 
project confirms the importance of further 
attention to (1) visualization as discipline, (2) the 
early stages of framing an idea and (3) 
optimizing that idea as well as to DES use case 
studies and future opportunities. 

N.B. The DES/VIZ team (P. Elfrey, J. 
Lagares, M. Fahmi, M Conroy and D. Mann) 
owes special debts of gratitude to David 
Miranda , Edgar Zapata , Martin Steele , Tracey 
Kickbusch, Scott Estes and Ben Bryant as well as 
Philip Michael, Luis Bares, Rebecca Mazzone, 
Bill Little, Matthew Verdier, Wyck Hebert, Lisa 
Zuber and Michelle Ruder. 

End Notes and References 

[1] . R. Kosslyn, 2006, Graphic Design 
for the Eye and Mind, Oxford University, USA 

[2] . P. Elfrey, M.Conroy, 2005, rev 2008 
Product Development Maturity Hierarchy 
adapted to DES/VIZ, NASA white paper. 

[3] . E. Raymond, 1997, 
http://tuxedo.org 

[4.] Herstatt, C, 2000, A Causal model 
of the Impact of the “fuzzy front end” on the 
success of new product development, Arbeiter, 
Germany 

[5]. R.G. Cooper and E.J. Kleinschmidt 
1998, new Product Performance: Keys to 
Success R&D Management Journal, July 1998, 
pp.281-298 Also Parry and Song, 2001, Portfolio 
Management for New Products , Perseus, NY 

[6.] T. Amabile, 1995, Assessing the 
Work Environment for Creativity , Harvard 
Business School, Cambridge, Mass 

[7]. J. Goguen, 1994, Requirements 
Engineering as the Reconciliation of Social and 
Technical Issues. In Requirements Engineering: 
Social and Technical Issues , edited with Marina 
Jirotka, Academic Press, pp. 165-199. 

[8.] Harvard University, Kennedy 
School of Public Administration, Ashe program 
survey, March 2008 


[9]. Sternman, J.D, 2004, All models 
are wrong: reflections on becoming a systems 
scientist, Systems Dynamics Review Vol. 18, 
No.4, John Wiley and Son Ltd., NY 

Biographies 

Priscilla Elfrey , NASA. SISO/Grand Challenges 
conference committee, SimSummit, Liophant & 
BOK advisory board member, produces 
simulation activities and projects and serves as 
broker for simulation partnering and related 
activity to strengthen modeling and simulation 
for Space. She was both project manager and 
mentor for the KSC DES/VIZ project. 

Jose Lagares , ASRC, an ESMD summer intern, 
is a graduate of the University of Puerto Rico 
(PR) at Ponce, he earned his M.S. in computer 
science from the Polytechnic University of PR, 
had previously interned at Goddard Space Flight 
Center and was student technical co-lead for this 
project. He created models, developed the tools 
and designed and programmed the simulation 
and, then, Alaskan Slope Regional Corporation 
(ASRC) hired him to work at KSC. 

Mona Fahmi, an ESMD summer intern, is 
currently a senior at the California State 
University in San Jose where she is a student 
leader of engineering groups and also works at 
the NASA Ames Research Center (ARC) on 
technology transfer. She was student co-lead on 
this project developed the presentation showing 
the steps needed to make the DES data visible 
and clarified the KSC ground processing path of 
flight hardware in ELLA. 

Michael Conroy , NASA Technical Advisor for 
IT/C modeling and simulation, leads NASA’s 
ESMD Data Processing and Visualization Team 
and initiated its Distributed Observer Network 
(DON) program. He served as customer for the 
DES/VIZ team 

David Mann Ph.D, (ASRC) supported DES/VIZ 
as the ASRC project manager and facilitated the 
transition from the problem as given: add 
visualization to DES to the development of the 
problem as understood: depict data within the 
discrete event simulation. 


